ACCURATE AND FAST STEREO VISION

Stereo vision from short-baseline image pairs is one of the most active research fields in computer vision. The estimation of dense disparity maps from stereo image pairs is still a challenging task and there is further space for improving accuracy, minimizing the computational cost and handling more efficiently outliers, low-textured areas, repeated textures, disparity discontinuities and light variations. This PhD thesis presents two novel methodologies relating to stereo vision from short-baseline image pairs: I. The first methodology combines three different cost metrics, defined using colour, the CENSUS transform and SIFT (Scale Invariant Feature Transform) coefficients. The selected cost metrics are aggregated based on an adaptive weights approach, in order to calculate their corresponding cost volumes. The resulting cost volumes are merged into a combined one, following a novel two-phase strategy, which is further refined by exploiting semi-global optimization. A mean-shift segmentation-driven approach is exploited to deal with outliers in the disparity maps. Additionally, low-textured areas are handled using disparity histogram analysis, which allows for reliable disparity plane fitting on these areas. II. The second methodology relies on content-based guided image filtering and weighted semi-global optimization. Initially, the approach uses a pixel-based cost term that combines gradient, Gabor-Feature and colour information. The pixel-based matching costs are filtered by applying guided image filtering, which relies on support windows of two different sizes. In this way, two filtered costs are estimated for each pixel. Among the two filtered costs, the one that will be finally assigned to each pixel, depends on the local image content around this pixel. The filtered cost volume is further refined by exploiting weighted semi-global optimization, which improves the disparity accuracy. The handling of the occluded areas is enhanced by incorporating a straightforward and time efficient scheme. The evaluation results show that both methodologies are very accurate, since they handle efficiently low-textured/occluded areas and disparity discontinuities. Additionally, the second approach has very low computational complexity. Except for the aforementioned two methodologies that use as input short-baseline image pairs, this PhD thesis presents a novel methodology for generating 3D point clouds of good accuracy from wide-baseline stereo pairs

A multi-modal dance corpus for research into interaction between humans in virtual environments

We present a new, freely available, multimodal corpus for research into, amongst other areas, real-time realistic interaction between humans in online virtual environments. The specific corpus scenario focuses on an online dance class application scenario where students, with avatars driven by whatever 3D capture technology is locally available to them, can learn choreographies with teacher guidance in an online virtual dance studio. As the dance corpus is focused on this scenario, it consists of student/teacher dance choreographies concurrently captured at two different sites using a variety of media modalities, including synchronised audio rigs, multiple cameras, wearable inertial measurement devices and depth sensors. In the corpus, each of the several dancers performs a number of fixed choreographies, which are graded according to a number of specific evaluation criteria. In addition, ground-truth dance choreography annotations are provided. Furthermore, for unsynchronised sensor modalities, the corpus also includes distinctive events for data stream synchronisation. The total duration of the recorded content is 1 h and 40 min for each single sensor, amounting to 55 h of recordings across all sensors. Although the dance corpus is tailored specifically for an online dance class application scenario, the data is free to download and use for any research and development purposes

Fusion of Sentinel-1 data with Sentinel-2 products to overcome non-favourable atmospheric conditions for the delineation of inundation maps

729Sentinel-1 data are an alternative for monitoring flooded inland surfaces during cloudy periods. Supervised classification approaches with a single-trained model for the entire image demonstrate poor accuracy due to confusing backscatter conditions of the inundated areas in relation with the prevailing land cover features. This study follows instead a pixel-centric approach, which exploits the varying backscatter values of each pixel through a time series of Sentinel-1 images to train local Random Forest classification models per 3×3 pixels, and classifies each pixel in the target Sentinel-1 image, accordingly. Reference training data is retrieved from the timely close Sentinel-2-derived inundation maps. This study aims to identify the furthest mean day difference between the target Sentinel-1 image and available Sentinel-2 high accurate inundation maps (kappa coefficient—k > 0.9) that allows for the estimation of credible inundation maps for the Sentinel-1 target date. Various combinations of Sentinel-2 and Sentinel-1 training datasets are examined. The evaluation for eight target dates confirms that a Sentinel-1 inundation map with a k of 0.75 on average can be generated, when mean day difference is less than 30 days. The increment of the considered Sentinel-2 maps allows for the estimation of Sentinel-1 inundation maps with higher accuracy

Increasing the accuracy of the space-sweeping approach to stereo reconstruction, using spherical backprojection surfaces

In this paper, interest is focused on the accurate and timeefficient stereo reconstruction, for the purpose of generating 3D animated scenes from multiple synchronized videos. The plane-sweeping approach is reviewed as relevant to the goal of time-efficiency, since its execution can be optimized on a GPU. A method compatible for optimization on the GPU is proposed as a more accurate alternative to plane sweeping and to the derived visibility computation. The method is compared to plane sweeping as to its accuracy, by evaluating the backprojected 3D model against independent views and using n-fold cross validation to estimate the Peak Signal to Noise Ratio (PSNR). Finally, the method’s output is casted integratable with multicamera stereo reconstruction frameworks. 1

Fast and Automatic Data-Driven Thresholding for Inundation Mapping with Sentinel-2 Data

Satellite data offer the opportunity for monitoring the temporal flooding dynamics of seasonal wetlands, a parameter that is essential for the ecosystem services these areas provide. This study introduces an unsupervised approach to estimate the extent of flooded areas in a satellite image relying on the physics of light interaction with water, vegetation and their combination. The approach detects automatically thresholds on the Short-Wave Infrared (SWIR) band and on a Modified-Normalized Difference Vegetation Index (MNDVI), derived from radiometrically-corrected Sentinel-2 data. Then, it combines them in a meaningful way based on a knowledge base coming out of an iterative trial and error process. Classes of interest concern water and non-water areas. The water class is comprised of the open-water and water-vegetation subclasses. In parallel, a supervised approach is implemented mainly for performance comparison reasons. The latter approach performs a random forest classification on a set of bands and indices extracted from Sentinel-2 data. The approaches are able to discriminate the water class in different types of wetlands (marshland, rice-paddies and temporary ponds) existing in the Doñana Biosphere Reserve study area, located in southwest Spain. Both unsupervised and supervised approaches are examined against validation data derived from Landsat satellite inundation time series maps, generated by the local administration and offered as an online service since 1983. Accuracy assessment metrics show that both approaches have similarly high classification performance (e.g., the combined kappa coefficient of the unsupervised and the supervised approach is 0.8827 and 0.9477, and the combined overall accuracy is 97.71% and 98.95, respectively). The unsupervised approach can be used by non-trained personnel with a potential for transferability to sites of, at least, similar characteristics.This study is supported and funded by the European Union’s Horizon 2020 research and innovation program under Grant Agreement No. 641762, ECOPOTENTIAL

A multimodal dance corpus for research into real-time interaction between humans in online virtual environments

International audienc

Ecosystem Services and Pressures in European Protected Areas: Divergent Views of Environmental Scientists and Managers

In the last decades intense anthropogenic pressure caused serious threats to ecosystems, leading to degradation of habitats and environmental quality, thereby increasing the risk of loss of ecosystem services (ES). Protected Areas (PA) may help to counterbalance degradation and associated loss of ES. In the EcoPotential project the state-of-art view was surveyed among environmental scientists and managers of PAs regarding the importance of various ecological, environmental, and socio-economic indicators for ES and pressures in their PA. Therefore, eight European PAs in mountainous areas, e.g. Kalkalpen and Gran Paradiso, and for comparison a few coastal PAs, e.g. Wadden Sea, were selected. Environmental scientists predominantly indicated abiotic and biotic factors as being most important for ES and pressures, whereas managers proportionally indicated socio-economic and cultural factors more often. Therefore, socio-economic and cultural factors (emphasised by managers) and abiotic and biotic factors (emphasised by scientists) need to be more integrated. Methods used worldwide for assessing the effectiveness of management in PAs may inspire the design of such an integrated framework. Moreover, in order to come to a concise list of variables for use in stakeholder engagement (incl. managers and policy-makers) these variables should be harmonised and preferably easy to measure, e.g. through Remote Sensing (RS). In our presentation we will show the different views of managers and scientists, how we may harmonise variables, and examples on how social (aesthetic) ES may be measured by RS